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Abstract 


In this study, we compare the teacher quality distributions in charter schools and traditional public 
schools, and examine mechanisms that might explain cross-sector differences in teacher effectiveness as 
measured by teacher value-added scores using school and teacher level data from Florida. We have 
three main findings. First, we find that teachers working in above-average poverty charter schools have 
significantly higher value-added scores compared to traditional public school teachers working in similar 
settings, which is mainly driven by the right tail of the value-added score distribution, yet we find no 
such differences in below-average poverty settings. Second, we find that cross-sector differences in 
observed teacher characteristics such as experience and educational attainment fail to explain any of 
the observed gaps in teacher effectiveness in higher-poverty settings. Instead, we find that differences 
in returns to experience on teacher productivity, which is significantly higher in the charter sector, 
explains most of the observed cross-sector effectiveness gaps. Third, we find considerable differences in 
teacher support and teacher influence on instructional policies and practices between charter schools 
and traditional public schools, which might help explain the higher returns to experience on teacher 
effectiveness as well as the observed effectiveness gaps between charter schools and traditional public 
schools serving disadvantaged students. 


Keywords: Charter Schools, Teacher Effectiveness, Teacher Value-Added, Urban School Districts 


1. Introduction 

School choice is a longstanding and highly debated reform strategy in the U.S. and one 
promoted by the Race to the Top legislation. School choice reforms extend the traditional Tiebout 
choice, in which residential choice implies school choice, by providing various alternatives to the 
household’s ‘neighborhood’ public school. These alternatives include private schools (via private school 
voucher programs), new publicly funded schools (such as charter and magnet schools), or other 
traditional public schools (via open enrollment programs which make out-of-boundary schools available 
to nonresidents). 

Over the past three decades, charter schools have become the most popular form of school 
choice, especially in urban school districts. Since the enactment of the first charter school legislation, the 
number of charter schools nationwide has soared from 2 in 1992 to 7,114 in 2015, making up roughly 8 
percent of all public schools. During the last decade, the number of students enrolled in public charter 
schools more than quadrupled from 300,000 students to 2.5 million students, accounting for 5.1 percent 
of all public school students and almost 15 percent of all public school students in urban school districts 
in 2013-14." As of 2010, 43 states and the District of Columbia had enacted a charter school law (NCES, 
2015). 

Economic theory may suggest that such market-based educational systems, where parents 
‘shop’ for schools, are efficiency-enhancing because they induce competition between schools and 
potentially produce better student-school matches. Furthermore, increasing parental choice can also be 
regarded as increasing equality of opportunity, since it serves to level the playing field in terms of access 


to high-quality education for disadvantaged students whose families cannot afford private school or 


‘In contrast, private school enrollment nationwide declined from roughly 6 million students to 5.4 million students 
during the same time frame. 


homes near better schools. Opponents, on the other hand, argue that school choice reforms hinder the 
progress of low-performing public schools by attracting the ‘best’ students and withholding much 
needed funds as students depart and enrollment numbers decline. As such, a great deal of empirical 
research has focused on school choice. 

The effectiveness with which charter schools raise student achievement has been studied at 
length. Observational studies of administrative data find that enrolling in a new charter school has a 
negative impact on student achievement growth, more so in newer schools.* Carruthers (2012a) shows 
that a small part of the maturation of new charter schools is due to accumulating faculty experience. In 
some settings, however, even relatively new charters compare very favorably to traditional public 
schools. On the other hand, recent studies have attempted to address the identification challenges by 
using the lottery-based admission policies to urban, oversubscribed charter schools and find large and 
positive impacts of charter attendance, yet the results for nonurban charters are mixed (Angrist et al., 
2013; Tuttle et al., 2013, Hoxby and Murarka 2009; Dobbie and Fryer, 2011; and Abdulkadiroglu et al., 
2011).? 

In this study, we investigate the degree to which differences in teacher quality explain the 
effectiveness of charter schools, and in doing so, we seek to bridge the gap between research on the 
effectiveness of charter schools and research demonstrating the profound importance of teachers in 
advancing student outcomes. Ex-ante, teacher quality is a likely factor for driving charter school 
effectiveness because researchers commonly recognize teachers as the most important schooling factor 


that has significant impact on students’ short-run test achievement and longer-term life outcomes, such 


2 See Bifulco & Bulkley (2008) and Gleason et al. (2010) for reviews. 

3 A related line of research examines the pathways by which charter schools affect students and teachers in 
traditional public schools. See Dee (1998), Hoxby (2002) and Booker et al. (2008) for empirical evidence that 
mainstream student performance improves in light of competition from choice schools. Fryer (2012) shows that 
some of the strategies employed by KIPP and “No Excuses” charter schools can be successfully implemented in 
traditional public schools. “No Excuses” philosophy in some urban charter schools has also been cited as a possible 
explanation to the disparity between urban and non-urban charter school performance (Angrist et al. 2011). 
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as college-going behavior and labor market earnings.’ For instance, Rockoff (2003) finds that a one 
standard deviation increase in elementary teacher quality increases reading scores by .20 standard 
deviations and math test scores by .24 standard deviations. Similarly, Chetty et al. (2014a) find that 
replacing an ineffective teacher with an average teacher in elementary school has a significant effect on 
the present value of students’ lifetime income. 

Despite general agreement on the importance of teacher quality, very little is known about 
charter teachers or disparities in teacher effectiveness between charter schools and traditional public 
schools (TPS) due to two important data limitations. First, in many locales (e.g., District of Columbia), 
charter schools are subject to different data reporting requirements than TPS, and therefore many 
educational data systems lack student-teacher linkages in charter schools that are necessary to estimate 
teacher value-added scores. Second, despite the rapid growth of charter schools nationwide, the size of 
the charter school sector in many states and school districts is insufficient for adequate statistical power 
in a comparison of teacher effectiveness distribution across sectors. In this study, we overcome these 
limitations by using teacher- and school-level data from Florida, which has one of the largest charter 
school systems along with one of the most sophisticated educational data systems in the United States.° 

Using these data, we compare the teacher value-added score distributions in charter schools 
and TPS, examine the extent to which these gaps are explained by observed teacher characteristics such 
as experience and educational attainment, and explore some of the mechanisms that might be 
contributing to these differences. Value-added scores are particularly well suited for this analysis 
because they have been commonly used as a measure of teacher effectiveness in research in the past 


decade (Goldhaber and Hansen, 2010; Hanushek et al., 2005; Harris and Sass, 2012; Koedel and Betts, 


4 See, for instance, Aaronson, Barrow, & Sander (2007), Goldhaber and Hansen (2013), Kane and Staiger (2008), 
and Hanushek and Rivkin (2010) for estimates of the effects of teachers on student test scores, and 

Chetty et al. (2014a) on the long-term effects of teachers. 

> In particular, Florida is currently ranked third behind California and Texas in terms of the number of charter 
schools and charter school enrollment in 2013-14 school year. 
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2011; Chetty et al., 2014a and 2014b), and are currently being used in teacher evaluations in several 
states and school districts, including Wisconsin, Florida, and DC. Research evidence suggests that value- 
added models perform well in isolating the contribution of a teacher’s impact on student achievement 
(e.g., Chetty et al. (2014b)).® 

We find that charter teachers working in above-average poverty settings have significantly 
higher value-added scores compared to TPS teachers serving similar student populations. We find no 
such cross-sector differences among schools serving more affluent students. The results also indicate 
that the cross-sector teacher effectiveness gaps in above-average poverty settings are mainly driven by 
the right tail of the teacher value-added distribution. That is, we find that the most effective teachers 
working in above-average poverty charter schools have significantly higher value-added scores than the 
most effective teachers working in above-average poverty TPS, while we find no such difference in the 
left-tail of the value-added distribution. These cross-sector teacher effectiveness gaps in urban, high- 
poverty settings might help explain the positive effects of attending an urban charter school on student 
achievement found in recent lottery-based studies. 

We also find that one of the mechanisms that might be contributing to these teacher 
effectiveness gaps in higher poverty settings is higher returns to experience on teacher productivity in 
the charter sector. In particular, our findings suggest no significant relationship between teacher value- 
added scores and experience in above-average poverty TPS. In contrast, charter teachers experience a 
large improvement in their value-added scores in the first 3-5 years of teaching. Our analysis of the 
2011-12 wave of Schools and Staffing Survey (SASS) for teachers in Florida also provide several factors 
that might be contributing to cross-sector teacher effectiveness gaps including higher levels of support 


charter teachers receive from school administration, other teachers, and parents; and higher levels of 


5 For a review of the literature on value-added modeling, see Koedel et al. (2015). 
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teacher influence on instructional policies and practices (e.g., course content and teaching techniques) 


in the charter sector, which might lead to higher levels of teacher engagement and productivity. 


2. Background 

Compared to the burgeoning research literature on charter effectiveness per se, very little is 
known about charter teachers or charter teacher effectiveness. To date, the body of research on charter 
teachers has been mostly limited to descriptive explorations of their observable characteristics such as 
attitudes, education, credentials, mobility, and pay.’ For instance, Stuit and Smith (2012) find that low 
union membership, low certification rates, and self-reported working conditions explain part of the fact 
that charter schools tend to have higher teacher turnover than traditional public schools. Jackson 
(2012) tests how charter school entry in North Carolina affects teacher labor markets, finding small 
increases in pay for traditional public school teachers and little impact on teacher turnover in traditional 
public schools.® 

To our knowledge, two recent studies are worth noting for their attention to teacher value- 
added in charter schools. Carruthers (2012b) shows that North Carolina teachers moving to charter 
schools tend to be less effective in terms of math and reading value added than teachers moving 
between quantitatively similar mainstream schools and that this disparity is even wider for teachers 


moving to urban, high-minority, or low-performing charter schools. Using elementary school teachers in 


‘ Survey evidence suggests that charter teachers work longer hours than traditional public school teachers (Malloy 
and Wohlstetter, 2003), have taken more math and science courses at the college level, but are no more or less likely 
to have earned a post-baccalaureate degree (Hoxby, 2002). Despite working longer hours and taking on 
responsibilities that are not normally expected of traditional public school teachers (like governance, hiring, and peer 
evaluation), charter teachers note that school autonomy and shared educational philosophies are among some of the 
non-pecuniary benefits of working in a charter school (Malloy and Wohlstetter 2003). Charter schools are generally 
free to exceed step-lane pay rates for teachers, but nonetheless, charter teachers tend to earn no more than their 
mainstream counterparts, and unlike mainstream school districts, charters do not necessarily pay teachers more for 
experience, licensure, or education (Podgursky & Ballou (2001); Hoxby (2002); Taylor (2005)). 

8 This suggests that teachers who leave traditional public schools for the charter sector likely would have changed 
schools or left teaching regardless of charter opportunities. 


fs) 


Florida, Cowen and Winters (2013) find no differential relationship between teacher attrition and 
teacher value added in charter schools. 

In this study, we extend the literature on charter teacher effectiveness by exploring the 
disparities in teacher value-added between charter schools and TPS using data from Florida. Since the 
enactment of the charter school legislation in 1996, the charter school sector in Florida has grown 
exponentially, with the number of charter schools increasing from 113 in 1999 to 646 in 2014, and 
charter school enrollment rising from 17,251 to 251,082 during the same time frame, placing Florida 
third only behind California and Texas along these two measures. As of 2015, almost 10 percent of all 
public school students in the state attend a charter school. Figure 1 maps the charter schools in Florida 
in 2011. Similar to other locales, charter schools in Florida are concentrated in less affluent, urban 
neighborhoods where the demand for alternative schooling options is higher. In particular, more than 
50 percent of all charter schools in Florida are located in five counties (out of 67) with large urban 
populations such as Broward, Dade, and Palm Beach all of which are part of Miami metropolitan area, 


Hillsborough (Tampa), and Orange (Orlando). In what follows, we detail the data used in our analysis. 


3. Data and Empirical Strategy 

Our primary measure of teacher effectiveness is the official teacher value-added scores 
compiled by Florida Department of Education (FDOE) as part of the state’s teacher evaluation system. 
Florida’s educator evaluation system, which was enacted in 2011, requires at least 50 percent of a 
teacher’s performance evaluation to be based on student performance in standardized tests.° For 


classroom teachers with students who take the Florida Comprehensive Assessment Test (FCAT), this 


° While individual districts have the flexibility to adjust the share of the student growth component in teacher 
ratings, Florida requires a minimum of 50 percent for all districts. The other component of the overall teacher ratings 
is the instructional practices portion which is primarily based on observations of teachers’ performance. These 
observations may be formal and informal, scheduled and unscheduled events, walkthroughs, meetings, and 
examination of materials that reflect the teacher’s work. 


“student growth” portion is calculated using the value-added model approved by the Student Growth 
Implementation Committee. The Florida value-added model is based on a covariate adjustment model 
that conditions a student’s current year test score on student demographics, prior test scores, and 
classroom characteristics, and teacher and school effects are estimated simultaneously in a random 
effects framework using the empirical Bayes estimator which shrinks teacher and school effects toward 
their population means in proportion to the degree of expected imprecision in the estimates. *° The final 
teacher value-added scores are then calculated by adding 50 percent of the empirical Bayes estimate of 
the unique school component to the empirical Bayes estimate of the unique teacher component. 

Following a lawsuit from the Florida Times-Union, FDOE released the value-added scores of all 
teachers in the state for years between 2011-12 and 2013-14. These publicly available data include the 
names of teachers and their schools along with the value-added scores in reading and math as well as 
combined scores that use test scores in both subjects, the standard errors associated with these value- 
added estimates, number of students included in the calculations by subject, and whether the teacher 
instructed an FCAT-related course. The reported value-added scores in the first year of the program 
(2011-12 school year) are annual scores that were estimated using classrooms in 2011-12 only, whereas 
the scores in the other two years are multi-year averages for which classrooms in multiple years were 
utilized (classrooms in the first two years were used to estimate 2012-13 teacher value-added scores 
whereas classrooms in all three years were used in 2013-14) when available. 

Using these data, we first compare the distribution of teacher value-added scores in TPS and 


charter schools. An important challenge in this exercise is that charter school teachers are significantly 


10 Tn particular, the Florida value-added model includes the following covariates: the number of subject-relevant 
courses in which the student is enrolled; two prior years of achievement scores (except for grade 4 where only one 
prior year achievement score is available); disability status; limited English proficiency; gifted status; student 
attendance; student mobility across schools; difference from modal age in grade; class size; and homogeneity of 
entering test scores in the class. For more information on Florida value-added model, see the Florida Value Added 
Model Technical Report, http://108.59.20.64/core/fileparse.php/7503/urlt/0072168- 
floridacomprehensiveassessmenttestvalue-addedmodeltechnicalreporti1112.rtf, accessed on 10/28/2016. 


ts 


more likely to be new teachers. Therefore, their value-added scores in the second and third years of the 
program are more likely to rely on fewer years of student performance compared to TPS teachers, which 
makes them more susceptible to random fluctuations. For instance, in 2012-13 school year, the value- 
added scores of 40 percent of charter school teachers relied on a single year of student performance 
compared to 26 percent for TPS teachers. While such random fluctuations are not expected to have an 
effect when comparing average value-added scores across sectors, they might increase the variance of 
value-added scores in charter schools and hence might have an effect when comparing the right and left 
tails of the value-added score distribution across sectors. To mitigate this effect, we focus on the first 
year of the program in our distribution analysis where the value-added scores of all teachers were 
calculated using one year of student performance. 

We supplement these teacher-level data with school characteristics from the Common Core of 
Data such as student composition, school poverty, and charter status; and measures of school 
performance such as the percentage of proficient students in reading and math from school 
accountability reports produced by FDOE for the state’s school accountability system. To investigate the 
extent to which differences in teacher value-added between charter schools and TPS can be explained 
by observed teacher characteristics, we also link these data to teacher characteristics such as experience 
and educational attainment from Miami-Dade County Public Schools, which houses more than 20 
percent of all charter schools in Florida. Finally, we make use of the 2011-12 wave of SASS for teachers 
in Florida to better understand why some charter teachers are more effective than comparable teachers 
in traditional public schools. 

Table 1 presents the school- and teacher-level characteristics of all public schools in Florida in 
2011-12 school year, broken down by school sector, charter school tenure, and school poverty as 
measured by the percent free-or-reduced-priced lunch (FRPL) eligible students in the school. Overall, 


charter schools in Florida have slightly higher percentages of students who are proficient, or have made 


adequate gains in reading and math, and students in charter schools are significantly less likely to be 
FRPL eligible and more likely to belong to a minority racial/ethnic group. These gaps remain almost 
unchanged when we compare TPS to mature charter schools that have been operational for at least 3 
years (which constitute about 70 percent of all charter schools in our sample)", or when we look at 
below average poverty (less than 60 percent of students FRPL eligible) or above average poverty 
schools. 

The second panel of Table 1 compares teacher experience and educational attainment across 
sectors in our Miami-Dade sample in 2011-12 for teachers who instructed an FCAT-related course. The 
findings indicate that charter school teachers have significantly less teaching experience and are 
significantly less likely to have advanced degrees compared to TPS teachers. In particular, 27 percent of 
charter teachers in our Miami-Dade sample have no prior teaching experience (compared to 13 percent 
among TPS teachers), and more than half of charter teachers have less than 3 years of experience 
(compared to 16 percent among TPS teachers). Once again, these gaps in teacher credentials remain 
unchanged when we exclude new charters, or when we break down the analysis by school poverty. 

To compare the teacher value-added distributions across school sectors and to explore the 
mechanisms that might be driving these differences, we follow a similar approach as in Sass et al. (2012) 
who compare the teacher value-added distributions in high-poverty versus lower-poverty schools. First, 
we compare teacher value-added scores in charter schools versus TPS at different points of the 
distribution (mean and 10°, 25, 50", 75", and 90" percentiles) as well as the dispersion of teacher 
value-added scores across sectors. Thus, not only do we investigate the effectiveness gaps between the 
average teachers in the two sectors, but we also examine differences between the highly effective and 
the ineffective teachers in the two school sectors. Second, we explore several mechanisms that might be 


driving these differences. In particular, we use Oaxaca-Blinder decomposition to break down the 


‘1 Results are similar when we use a threshold of 5 years to identify mature charter schools. 
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difference in average teacher effectiveness across sectors into three components: (1) differences in 
effectiveness due to differences in observed teacher attributes; (2) differences in effectiveness due to 
differences in returns to these attributes (e.g., whether the returns to experience in one sector is larger 
than the other); and (3) differences in effectiveness due to the interaction between attributes and their 
returns. Further, we examine factors such as differences in teacher support, working conditions and job 


satisfaction across sectors in Florida using the teacher responses in the 2011-12 wave of SASS. 


4. Results 


4.1. Differences in Value-Added Score Distributions across Sectors 


Table 2 presents the means and standard deviations of value-added scores along with value- 
added scores at the 10°", 25, 50", 75", and 90" percentiles of the distribution in math (first panel), in 
reading (second panel), and combined scores (third panel) by school sector. This analysis includes all 
teachers who instructed an FCAT-related course in 2011-12. The second and third columns of the table 
provide the statistics for all TPS and charter teachers in Florida public schools in 2011-12, the fourth 
column presents the differences between the two groups, the fifth column restricts the sample to 
mature charter schools, and the last column provides the differences between all TPS teachers and 
teachers working in mature charter schools. 

We find small and statistically insignificant differences favoring charter teachers in the average 
reading and math value-added scores between the two sectors. The difference in average combined 
value-added scores is slightly higher, yet is only marginally significant at conventional levels. The 
dispersion of value-added scores in the charter sector is significantly higher in math, which is mainly 
driven by the right tail of the value-added distribution in charter schools. That is, the most effective 
teachers in the charter sector have significantly higher value-added scores than the most effective 


teachers in TPS. For example, charter students assigned to a math teacher whose value-added score falls 
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on the 90" percentile of the charter teacher value-added distribution perform 3.6 percent of a standard 
deviation better in math tests than similar TPS students assigned to a math teacher whose value-added 
scores falls on the 90" percentile among TPS teachers. Similar results emerge when we use combined 
value-added scores, yet the findings are much weaker in reading. 

We observe similar patterns that are even stronger when we restrict the charter teacher sample 
to those who teach in mature charter schools. Once again, highly effective teachers in mature charter 
schools have significantly higher value-added scores in reading and math compared to highly effective 
TPS teachers, while we find no such differences at the lower end of the value-added distribution. Figure 
2 presents a more complete picture by portraying value-added scores in mature charter schools and TPS 
at each percentile of the value-added distribution in each sector, and shows how teacher value-added 
scores in mature charter schools diverge from the value-added scores of TPS teachers as we move from 
the lower end of the distribution to the higher end. 

A key goal for this study is to examine if differences in teacher quality between urban and 
nonurban settings can be explained by the distribution of teacher quality. As discussed above, several 
lottery-based studies have recently found significant positive effects of attending an urban charter 
school on student outcomes, yet these benefits do not necessarily carry over to charter schools 
operating in more affluent non-urban settings. For instance, Abdulkadiroglu et al. (2011) shows that 
students who won admission lotteries and subsequently attended urban charter schools in 
Massachusetts perform significantly better on reading and math tests compared to similar students who 
lost the lottery, yet no such effect exists in non-urban charter schools that typically serve more affluent 
students. One possible mechanism that might explain this finding is that charter teachers serving 
disadvantaged populations are more effective in raising student achievement than TPS teachers serving 


similar populations, yet this effectiveness gap does not exist in more affluent neighborhoods. 
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Table 3 explores this possibility by breaking down our analysis by school poverty where columns 
2-4 compare value-added distributions between TPS and mature charter teachers in low-poverty 
settings, whereas columns 5-7 examine schools serving less affluent students. Analogous to Figure 2, 
Figure 3 presents the value-added scores at each percentile broken down by school sector and school 
poverty. The findings suggest that the differences in teacher performance across sectors on the higher 
end of the value-added score distribution are entirely driven by schools serving disadvantaged students. 

In particular, among schools serving more affluent students, we find no significant difference in 
the distribution of value-added scores of charter teachers and the distribution of value-added scores of 
TPS teachers. In contrast, among schools serving disadvantaged students, the charter teacher value- 
added score distribution in each subject diverges significantly from the TPS teacher value-added 
distribution as we move from the left tail of the distribution to the right. Specifically, looking at 
combined value-added scores, we find no significant differences between the least effective charter 
teachers (teachers whose combined scores fall on the 10 or the 25" percentile of charter teacher 
distribution) and the least effective TPS teachers. This difference increases to 2.5 percent of the 
standard deviation and is marginally significant when we compare the median teachers in the two 
sectors. The most effective teachers at above-average poverty charter schools (teachers whose 
combined scores fall on the 75" or the 90" percentile of charter teacher distribution) have significantly 
higher value-added scores than the most effective TPS teachers serving disadvantaged schools. Overall, 
the average charter teacher at above average poverty charter schools has a combined value added score 
that is 3.1 percent of the standard deviation higher than the average TPS teacher in a similar school. 

An important concern when comparing teacher value-added scores across schools is the 
possibility that the measured differences might be driven by common school effects that are unrelated 
to teacher quality instead of differences in true teacher effectiveness. This is also the case in our study 


where differences in school leadership or policies and practices across sectors (e.g., curriculum) might 
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explain the observed differences in value-added scores of charter and TPS teachers in higher poverty 
settings, even though no such differences exist in true teacher effectiveness. While it is not possible to 
disentangle teacher and school effects and test for this hypothesis directly, we follow a similar approach 
as in Sass et al. (2012) and present indirect evidence. 

In particular, we make use of teachers with value-added scores who served in both higher 
poverty mature charter schools and higher poverty TPS during the three years between 2011-12 and 
2013-14. If differences in school effects are indeed driving differences in observed teacher value-added 
scores, one would expect sector switchers to have significantly higher value-added scores in the higher 
poverty charter school setting. The main challenge in this exercise is that, as documented in Cowen and 
Winters (2013), there are very few sector switchers in Florida. Therefore, to maximize sample size, we 
focus on combined value-added scores using both reading and math teachers. Over the three years we 
examine, there were 66 unique teachers with valid combined value-added scores who switched sectors 
ina higher poverty setting once. Of these teachers, 38 moved from a charter school to a TPS and 28 
moved from a TPS to a charter school. 

Table 4 presents the combined value-added scores of these teachers before and after the 
switch, broken down by the direction of the move. There are several findings worth highlighting. First, 
teachers who moved from a charter school to a TPS have significantly lower value-added scores prior to 
the switch. Second, neither type of switchers experienced a significant change in their value-added 
scores following the move. Finally, we find no significant differences in the within-teacher effects across 
school sectors, providing evidence against the hypothesis that our results are driven by school effects 


that are higher in the charter sector. 


4.2. Mechanisms behind Discrepancies in Teacher Effectiveness across Sectors 


In this section, we examine several mechanisms that might be contributing to the observed 


differences in teacher effectiveness between mature charter schools and TPS serving disadvantaged 
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student populations. Table 5 presents the Oaxaca-Blinder decomposition of cross-sector differences in 
value-added scores, and explores the extent to which differences in teacher experience and educational 
attainment explain effectiveness gaps. For this exercise, we use Miami-Dade teachers between 2011-12 
and 2013-14 for whom we observe experience and educational attainment, and once again rely on 
combined value-added scores to maximize sample size. 

There are two main takeaways from this analysis. First, similar to our statewide findings, above- 
average poverty charter teachers in Miami-Dade have significantly higher combined value-added scores 
than TPS teachers serving less affluent students in the district. Second, differences in returns to 
observed teacher attributes, experience in particular, explain the entire effectiveness gap between 
above-average poverty charter and TPS teachers. In contrast, differences in observed attributes or the 
interactions with their returns explain virtually none of the observed differences in teacher value-added 
scores. 

Table 6 further examines the returns to teacher characteristics on teacher effectiveness in TPS 
versus charter schools in higher poverty settings. Similar to Sass et al. (2012), we find no significant 
relationship between teacher experience and effectiveness in above-average poverty TPS schools. In 
contrast, there are significant returns to experience on teacher productivity in charter schools serving 
similar student populations, especially in the first 5 years of teaching. In particular, charter teachers with 
no prior teaching experience have combined value-added scores that are 3 percent of the standard 
deviation lower than teachers with 1 to 2 years of experience (although this difference is not statistically 
different from zero), and this gap further widens to 6 percent when comparing novice teachers to 
charter teachers with 3 to 5 years of experience. 

There are several factors that might explain the higher returns to experience on teacher 


effectiveness in charter schools. For instance, charter schools might be better at filtering out less 
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effective early-career teachers as a result of less restrictive personnel policies.’* An important limitation 
of our data that prevents us from exploring this possible mechanism is that we only observe teachers 
with value-added scores. Therefore, we are unable to distinguish between cases where a teacher is no 
longer in the sample because he/she left the teaching workforce and cases where a teacher is no longer 
in the sample because he/she teaches an untested grade in the same school. When we examine teacher 
mobility conditional on staying in the sample (i.e., teacher moving to another school and teaching a 
tested grade), we find that teachers who left an above-average poverty charter school had significantly 
lower combined value-added scores before the move compared to teachers left a TPS serving similar 
students (-0.12 versus -0.07).*3 

Another factor that might explain the higher returns to experience in charter schools (and the 
effectiveness gaps between charter schools and TPS serving disadvantaged students) is differences in 
school climate and school policies and practices between sectors. Table 7 compares the charter and TPS 
teacher responses in the 2011-12 wave of SASS in Florida along four dimensions: (1) teacher support 
(first panel); (2) teacher job satisfaction (second panel); (3) teacher working conditions (third panel); and 
(4) the influence of teachers on school policies and practices (fourth and fifth panels). We are unable to 
break down this analysis by school poverty due to small sample sizes in SASS, and present the cross- 
sector differences along these measures using all surveyed schools in Florida in 2011-12 school year. 

The findings reported in the first panel indicate that charter teachers receive more support 
overall, especially from parents and students. In particular, charter teachers are slightly more likely to 
agree that the school administration is supportive and encouraging, that they are recognized for a job 


well done, and that there is cooperative effort among staff compared to TPS teachers in Florida. The 


? According to the 2011-12 wave of SASS, 13 percent of charter school principals in Florida reported personnel 
policies as a barrier to dismissal of poor performing teachers, in stark contrast to 69 percent of TPS principals. 

13 Tn particular, over the three years we examine, 44 teachers left an above-average poverty mature charter school, 
and 34 teachers left an above-average poverty TPS. 
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cross-sector differences in teacher responses are significantly larger when asked about whether 
teachers receive a great deal of support from parents (67 percent of charter teachers agree compared to 
52 percent of TPS teachers); or whether lack of parental involvement is a problem (39 percent of charter 
teachers agree compared to 62 percent of TPS teachers); or whether student unpreparedness is a 
problem at school (44 percent of charter teachers agree compared to 66 percent among TPS teachers).** 
Higher levels of support in charter schools might prevent teacher “burn-out”, resulting in larger 
increases in productivity over time compared to TPS teachers working in high-poverty settings. 

Higher levels of support and acknowledgement might also lead to higher levels of job 
satisfaction among charter teachers, of which we find modest evidence. Specifically, we find that charter 
teachers are slightly more likely to be satisfied with their salary (although salary satisfaction is low in 
both sectors), less likely to be worried about their job security, more likely to be satisfied with teaching 
at their current schools, less likely to leave teaching for higher pay, and less likely to report that they 
have less enthusiasm than when they first began teaching, although cross-sector differences in teacher 
responses in almost all cases are small. Aside from higher levels of support, these differences in job 
satisfaction might be driven by the fact that charter teachers have significantly less teaching experience 
than TPS teachers if teacher optimism dissipates over time, especially among teachers working in more 
challenging environments. 

Charter teachers also have slightly better working conditions compared to TPS teachers. 
Significantly fewer charter teachers report student misbehavior or tardiness as interfering with teaching 
activities (30 percent versus 37 percent for misbehavior, 20 percent versus 42 percent for tardiness), or 


report that routine duties interfere with teaching (69 percent versus 78 percent); a higher percentage of 


14 Cross-sector differences in parental support might be driven by the fact charter parents make an active choice to 
send their children to charter schools, which might be considered as a sign of higher educational motivation 
compared to TPS parents. 
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charter teachers state that they have necessary materials available for teaching (82 percent versus 71 
percent). 

Another interesting cross-sector difference in working conditions is that charter teachers are 
significantly more likely to agree that content standards have positive influence on teaching (63 percent 
versus 41 percent) compared to TPS teachers. One possible explanation behind this finding is that 
charter teachers in Florida have significantly more influence on school policies and practices, especially 
in terms of instructional policies and practices, which might lead to higher rates of teacher buy-in for 
course content in charter schools. In particular, compared to TPS teachers, charter teachers have 
significantly more influence on selecting materials for classroom (52 percent versus 35 percent), content 
taught in classroom (56 versus 45 percent), establishing curriculum (62 versus 42 percent), teaching 
techniques used in classroom (95 versus 82 percent), and evaluating and grading students (96 versus 89 
percent). Charter teachers also have larger control over disciplining students and evaluating other 
teachers, yet we find no significant cross-sector differences in teacher influence on the content of 
professional development, hiring new full time teachers, or school budget. Higher levels of control over 
instructional practices and course content among charter teachers might lead to higher levels of teacher 
engagement and motivation, which might in turn explain the higher returns to experience on teacher 
effectiveness and the observed discrepancies in teacher value-added scores between charter schools 


and TPS serving disadvantaged students. 


5. Conclusions 

In this study, we compare the teacher quality distributions in charter schools and TPS, and 
examine mechanisms that might explain cross-sector differences in teacher effectiveness as measured 
by teacher value-added scores. We have three main findings. First, we find no significant difference in 


teacher quality distribution between lower-poverty charter schools and TPS. In contrast, we find that 
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teachers working in above-average poverty charter schools have significantly higher value-added scores 
compared to TPS teachers working in similar settings. These differences in teacher effectiveness are 
mainly driven by the right tail of the value-added score distribution. That is, while we find no significant 
cross-sector differences between the least effective teachers in the two sectors, the most effective 
teachers in above-average poverty charter schools have significantly higher value-added scores 
compared to the most effective TPS teachers in similar settings. 

Second, we find that cross-sector differences in observed teacher characteristics such as 
experience and educational attainment fail to explain any of the observed gaps in teacher effectiveness 
in higher-poverty settings. Instead, these gaps are primarily driven by differences in returns to these 
attributes — experience in particular — on teacher productivity. In particular, the results indicate no 
significant relationship between teacher experience and effectiveness in above-average poverty TPS 
schools whereas the value-added scores of teacher improve considerably in the first 3 to 5 years of 
teaching in similar charter schools. 

Third, using the responses of Florida public school teachers in the 2011-12 wave of SASS, we find 
significant cross-sector differences in the level of support teacher receive (especially from parents), 
which might help explain the higher returns to experience on teacher effectiveness as well as the 
observed effectiveness gaps between charter schools and TPS serving disadvantaged students. In 
particular, higher levels of support might prevent teacher “burn-out”, resulting in larger increases in 
productivity over time compared to TPS teachers working in high-poverty settings. We also find that 
charter teachers have significantly more influence on classroom policies and practices including course 
content and teaching techniques, which might lead to higher teacher productivity through higher levels 
of engagement and motivation. 

Our study has several important implications for public policy. If teacher effectiveness gaps are 


mainly driven by charter schools being better at filtering out less effective teachers as a result of less 
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restrictive personnel policies, which we are unable to test directly due to data limitations yet find some 
indirect evidence for, less restrictive personnel policies might also be preferred in above-average 
poverty TPS schools. Similarly, if cross-sector differences in teacher influence on school policies is the 
main driver of these effectiveness gaps, such policies might be adopted in TPS to increase teacher 
motivation and prevent burn-out. In contrast, no public policy will likely remedy the teacher 
effectiveness gaps between charter schools and TPS in higher poverty settings if higher levels of parental 
involvement in charter schools, which might be naturally occurring as more educationally motivated 
parents send their children to these schools, create a better working environment for charter teachers 
and increase their effectiveness. 

Our findings might also shed light on prior research on charter schools. In particular, recent 
studies that make use of admission lotteries in oversubscribed charter schools find positive effects of 
attending an urban charter school on student outcomes — a finding that does not necessarily persist in 
non-urban settings. Cross-sector differences in teacher effectiveness in urban settings that do not carry 
over to more affluent non-urban settings, of which we find evidence in this study, might explain the 


observed benefits of attending charter schools in higher-poverty urban neighborhoods. 
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Figure 1 — Charter Schools in Florida, 2011-12 School Year 
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Figure 2 — Value-Added Scores by Percentile, Subject, and School Sector, 2011-12 School Year 
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Notes: Each figure displays the value-added scores at each percentile of the distribution in traditional 
public schools and charter schools that have been operational for at least 3 years by subject. 
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Figure 3 — Value-Added Scores at Various Percentiles, by School Sector and Poverty, 2011-12 
School Year 
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Notes: Each figure displays the value-added scores at each percentile of the distribution in traditional 
public schools and charter schools that have been operational for at least 3 years by subject and school 
poverty. 
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Table 1 — School and Teacher Characteristics in Charter Schools and TPS in Florida, 2011-12 School Year 


Below Average Poverty Above Average Poverty 
Mature Mature Mature 
Schools - TPS All Charters Charters TPS Charters TPS Charters 
% proficient in reading 0.571 0.600" 0.604" 0.687 0.690 0.486 0.485 
(0.156) (0.182) (0.184) (0.118) (0.148) (0.123) (0.161) 
% proficient in math 0.578 0.588 0.598" 0.688 0.661" 0.496 0.513 
(0.151) (0.191) (0.189) (0.113) (0.165) (0.122) (0.187) 
% gains in reading 0.673 0.685" 0.689" 0.702 0.708 0.651 0.664* 
(0.0788) (0.105) (0.103) (0.0709) (0.0948) (0.0772) (0.109) 
% gains in math 0.664 0.671 0.681" 0.703 0.695 0.635 0.662" 
(0.111) (0.160) (0.153) (0.107) (0.144) (0.106) (0.163) 
% FRPL eligible 0.599 0.495°™" 0.508°™" 0.362 0.335°™" 0.795 0.775** 
(0.258) (0.257) (0.259) (0.170) (0.163) (0.110) (0.111) 
% white 0.424 0.359" 0.356" 0.567 0.462" 0.305 0.193" 
(0.271) (0.287) (0.286) (0.218) (0.266) (0.253) (0.235) 
% black 0.271 0.276 0.272 0.175 0.177 0.350 0.417°™" 
(0.254) (0.288) (0.286) (0.175) (0.196) (0.281) (0.337) 
% Hispanic 0.250 0.315" 0.323" 0.191 0.304°™" 0.298 0.353""" 
(0.227) (0.285) (0.284) (0.167) (0.253) (0.256) (0.324) 
Number of schools 3,781 633 419 1,536 241 2,245 178 
Teachers (Miami-Dade)- 
Experience (in years) 10.45 2.157"" 2.341" 11.24 2.332"" 10.25 2.349" 
(8.475) (2.802) (2.864) (8.535) (2.823) (8.449) (2.903) 
No experience 0.128 0.269" 0.244" 0.102 0.233" 0.134 0.253" 
(0.334) (0.444) (0.430) (0.303) (0.423) (0.341) (0.435) 
Experience: 1-2 years 0.0393 0.268°"" 0.270°™ 0.0279 0.282" 0.0422 0.260" 
(0.194) (0.443) (0.444) (0.165) (0.450) (0.201) (0.439) 
Experience: 3-5 years 0.146 0.164 0.177" 0.140 0.197" 0.147 0.161 
(0.353) (0.370) (0.382) (0.348) (0.399) (0.355) (0.368) 
Experience: 6-12 years 0.317 0.0910" 0.0996" 0.329 0.091°™" 0.313 0.107" 
(0.465) (0.288) (0.300) (0.470) (0.288) (0.464) (0.309) 
Experience: 13+ years 0.370 0.195"™" 0.201°™ 0.398 0.184"™" 0.362 0.214°" 
(0.483) (0.397) (0.401) (0.490) (0.388) (0.481) (0.410) 
Advanced degree 0.362 0.158" 0.160°™ 0.365 0.201" 0.361 0.128" 
(0.481) (0.365) (0.367) (0.482) (0.401) (0.480) (0.334) 
Number of teachers 5,491 675 618 1,108 285 4,384 334 


Notes: Standard deviations in parentheses. *, “, and “* indicate that the average charter school or charter teacher attribute is different than that of 
TPS or TPS teachers at 10, 5, or 1 percent significance level. 
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Table 2 — Means and Standard Deviations of Teacher Value-Added, and Value-Added Scores at 
Various Percentiles, by School Sector, 2011-12 School Year 


Charter Mature Charter 
TPS Schools Difference Schools Difference 
Math - 
Mean 0.011 0.023 0.012 0.034 0.023° 
Standard deviation 0.338 0.355 0.017" 0.355 0.017" 
10" percentile -0.399 -0.399 0.0001 -0.390 0.009 
25" percentile -0.204 -0.190 0.014 -0.182 0.022" 
50" percentile 0.001 0.007 0.006 0.018 0.017 
75" percentile 0.216 0.230 0.014 0.244 0.028 
90" percentile 0.433 0.469 0.036° 0.472 0.039°™ 
Number of teachers 23,482 1,845 1,574 
Reading - 
Mean -0.003 0.005 0.008 0.009 0.012 
Standard deviation 0.227 0.227 0.0001 0.228 0.001 
10" percentile -0.247 -0.241 0.006 -0.237 0.01 
25" percentile -0.111 -0.112 -0.001 -0.108 0.003 
50" percentile -0.002 0.009 0.011 0.011 0.013" 
75" percentile 0.109 0.125 0.016° 0.131 0.022" 
90" percentile 0.237 0.253 0.016 0.257 0.020 
Number of teachers 35,296 2,504 2,130 
Combined - 
Mean 0.0001 0.017 0.017" 0.023 0.023""" 
Standard deviation 0.267 0.277 0.010" 0.278 0.011™ 
10" percentile -0.311 -0.299 0.012 -0.295 0.016 
25" percentile -0.148 -0.143 0.005 -0.132 0.016 
50" percentile -0.002 0.009 0.011 0.016 0.018" 
75" percentile 0.146 0.166 0.020" 0.175 0.029"™" 
90" percentile 0.309 0.341 0.032" 0.343 0.034" 
Number of teachers 44,537 3,291 2,816 
Note: *, “, and “” indicate that the statistic for the charter teacher value-added score distribution is 


different than that of TPS teachers at 10, 5, or 1 percent significance level. 
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Table 3 — Means and Standard Deviations of Teacher Value-Added, and Value-Added Scores at 
Various Percentiles, by School Sector and Poverty, 2011-12 School Year 


Below Average Poverty Above Average Poverty 
Mature Charter Mature Charter 
TPS Schools Difference TPS Schools Difference 
Math - 
Mean 0.039 0.037 -0.002 -0.010 0.030 0.040° 
Standard deviation 0.325 0.345 0.020" = 0.345 0.373 0.028" 
10" percentile — -0.351 -0.370 -0.019 -0.427 -0.408 0.019 
25" percentile -0.166 -0.179 -0.013 -0.230 -0.184 0.046" 
50" percentile 0.025 0.020 -0.005 -0.020 0.016 0.036 
75" percentile 0.231 0.245 0.014 0.202 0.242 0.040 
90" percentile 0.447 0.478 0.031 0.422 0.469 0.047" 
Number of teachers 9,993 1,016 1,574 558 
Reading - 
Mean 0.021 0.019 -0.002 -0.023 -0.010 0.013 
Standard deviation 0.242 0.223 -0.019 0.211 0.235 0.024 
10™ percentile — -0.235 -0.207 0.028 -0.256 -0.280 -0.024 
25" percentile —_-0.093 -0.099 -0.006 -0.126 -0.127 -0.001 
50" percentile 0.017 0.017 0.0001 -0.017 0.002 0.019° 
75" percentile 0.136 0.137 0.001 0.089 0.121 0.032" 
90" percentile 0.279 0.258 -0.021 0.201 0.254 0.053" 
Number of teachers 16,340 1,376 18,956 754 
Combined - 
Mean 0.027 0.031 0.004 -0.022 0.009 0.031" 
Standard deviation 0.273 0.272 -0.001 0.262 0.287 0.025"™ 
10" percentile — -0.281 -0.269 0.012 -0.332 -0.331 0.001 
25" percentile -0.124 -0.123 0.001 -0.169 -0.153 0.016 
50" percentile 0.021 0.023 0.002 -0.021 0.004 0.025" 
75" percentile 0.173 0.177 0.004 0.123 0.173 0.050" 
90" percentile 0.345 0.347 0.002 0.281 0.336 0.055°™ 
Number of teachers 20,384 1,791 24,153 1,025 


3K 


Note: *, ™, and “* indicate that the statistic for the charter teacher value-added score distribution is 
different than that of TPS teachers at 10, 5, or 1 percent significance level. 
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Table 4— Combined Value-Added Scores of Sector Switchers in Higher-Poverty Setting, Before 
and After the Switch, by the Direction of Switch 


Before the After the Difference 
switch switch 
Charter school to TPS -0.084 -0.081 0.003 
TPS to charter school -0.024 -0.027 -0.003 
Difference (charter to TPS vs. TPS to charter) -0.060 -0.054 
All switchers (charter vs. TPS) 0.009 


Note: The analysis sample includes teachers who worked in a charter and a traditional public school with 
above-average poverty level between 2011-12 and 2013-14 school years with non-missing combined 
value-added scores. We also restrict the sample to teachers who switch sectors only once during this time 
frame. The numbers indicate the average combined value-added scores of sector switchers before and 
after the switch. 
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Table 5 — Oaxaca-Blinder Decomposition of Cross-Sector Differences in Combined Teacher 
Value-Added Scores, Miami-Dade Sample, Above Average Poverty Schools 


Mean predicted charter teacher score 
Mean predicted TPS teacher score 
Difference (charter teachers vs. TPS teachers) 
Due to differences in... 
Teacher characteristics 
Returns to teacher characteristics 


Interaction of returns and characteristics 


0.039°" 
(0.010) 

0.017" 
(0.003) 

0.022"* 
(0.010) 


-0.001 
(0.015) 
0.034" 
(0.011) 
0.012 
(0.015) 


Note: The analysis sample includes teachers who worked in Miami-Dade County public school between 
2011-12 and 2013-14 school years with non-missing combined value-added scores. Teacher 
characteristics included in the analysis are indicators for teachers with 1-2 years of experience, 3-5 years 
of experience, and 6 or more years of experience, and an indicator for teachers with advanced degrees. 
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Table 6 - Combined Teacher Value-Added Scores and Teacher Characteristics, Miami-Dade 
Sample, Above Average Poverty Schools, by School Sector 


TPS Teachers Charter teachers 

Experience 1-2 years 0.0002 0.032 
(0.012) (0.026) 
Experience 3-5 years 0.020 0.057" 
(0.012) (0.029) 

Experience 6+ years 0.016" 0.027 
(0.009) (0.032) 

Advanced degree 0.004 -0.009 
(0.006) (0.033) 

N 13,553 1,149 


Note: Standard errors in parentheses. *, “, and “ indicate statistical significance at 10, 5, or 1 percent 
significance levels respectively. Feasible Generalized Least Squares (FGLS) is used to account for the 
error in value-added scores. 
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Table 7 — School Climate and Teacher Attitudes in Florida, by School Sector, 2011-12 Wave of 
Schools and Staffing Survey 


TPS Teacher Charter Teachers 


Teacher support 


School administration is supportive and encouraging 0.883 0.903 
Staff members are recognized for a job well done 0.754 0.840 
There is cooperative effort among staff 0.822 0.910 
Receive a great deal of support from parents 0.522 0.667 
Lack of parental involvement is a problem 0.624 0.390 
Student unpreparedness is a problem at school 0.660 0.437 
Teacher job satisfaction 
Satisfied with salary 0.219 0.251 
Teachers worry about the security of their jobs 0.634 0.595 
Teachers are satisfied with teaching 0.894 0.907 
Teachers are satisfied teaching at the school 0.724 0.780 
Teachers would leave teaching for higher pay 0.412 0.352 
Teachers have less enthusiasm than when first began 0.424 0.346 
Teacher working conditions 
Student misbehavior interferes with teaching 0.370 0.304 
Student tardiness interferes with teaching 0.420 0.193 
Routine duties interfere with teaching 0.771 0.687 
Have necessary materials available 0.713 0.818 
Content standards have positive influence on teaching 0.412 0.632 
Teachers have control over... 
selecting materials for classroom 0.346 0.524 
content taught in the classroom 0.454 0.557 
selecting teaching techniques used in classroom 0.818 0.949 
evaluating and grading students 0.888 0.961 
disciplining students 0.877 0.938 
High influence of teachers in... 
setting performance standards for students 0.463 0.580 
establishing curriculum 0.424 0.617 
content of professional development 0.428 0.439 
evaluating other teachers 0.140 0.205 
hiring new full time teachers 0.169 0.188 
setting discipline policy 0.374 0.420 
school budget 0.182 0.171 


Notes: The numbers presented in the first three panels indicate the percentage of teachers who somewhat 
or strongly agree with each listed item whereas the numbers in fourth and fifth panels indicate the 
percentage of teachers who think that teachers have moderate or great deal of control or influence on each 
listed item. 
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